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Introduction 


A random sample of students who took the ACT test in either April or June of 2016 were 
invited to participate in an online survey via an email invitation (N = 35,047*). To study 
whether mailing post card invitations to students improves response rates, a portion of 
these test-takers were randomly assigned to be sent a post card. In addition, students 
who were sent a post card were randomly assigned to either receive one that pictured a 
female student or one that pictured a male student. All post cards described the research 
study and provided the student with the survey link. All groups received similar invitation 
via email. Three research questions were investigated. 
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schools. The results indicate that sending a post card to invite students to participate in an online 


survey did not improve response rates appreciably (Figure 1). A total of 5.3% of students 
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students who were not sent a post card. This difference was not statistically nor practically 


Figure 1. Response Rates, by Post Card Status 


significant. 


1 This is part of a larger study (N = 58,559) seeking to understand the impact that financial incentives and post card invitations 
had on response rates. Only the effects of post card invitations are provided in this report. Students were randomly assigned to 
receive either the post card or the incentive, but not both. 
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Sending postcards may be ineffective as well as relatively expensive, as it cost 
approximately $7 per post card, per completed survey compared to essentially $0 to send 


an email. 


Research Question 2: Does the picture on the post card have an impact on 


response rates? 


Figure 2 shows that the picture on the post card did not have an impact on response 
rates. Of those students who were sent a post card, both male and female post card 
images were associated with 5.3% response rates. In addition, the picture on the post 
card did not have a differential impact on response rates relative to the overall post card 


response rate. 
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Figure 2. Response Rates, by Post Card Picture 


Research Question 3: Does the picture on the post card have a differential impact 


on response rates based on the gender of the survey respondent? 


Using a picture of a male student on the post card invitation had a differential impact on 
male and female survey respondents (See Figure 3). Providing female respondents with a 
male-picture post card resulted in a slightly higher response rate (6.7%) than providing 
them with a female-picture post card (6.3%) or no post card at all (6.0%). The opposite 
effect was found for male survey respondents. Males, when provided with a male-picture 
post card, responded at the lowest rate (3.5%), which was even lower than when no post 
card at all was provided (3.8%). Although these differential effects by gender are not 
statistically significant nor have any magnitude of an effect, it might be interesting to 
replicate this study, perhaps with a stronger stimulus such as an image on the header of 


each page of the survey itself. 
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Respondents 
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Figure 3. Response Rates, by Gender of the Survey Respondent and Post Card 
Category 


It is worth noting that Figure 3 also shows that female students were more likely to 
complete the survey than were male students. Regardless of whether a post card was 
received or the gender of the picture on the post card, female students were statistically 
more likely to respond (6.0 — 6.7%, depending on the experimental group) than their male 
counterparts (3.5% — 4.1%). 


